SUMMARY 



This project was an evaluation-research training program. The prime 
contractor was the Research Council of the Great City Schools , and North- 
western University, the subcontractor. 

The purpose of the project was to involve local program evaluators in 
training activities that would increase competencies toward evaluation and 
the consequent improvement of instructional programs at the local level. 
Northwestern University served as the subcontracting agency and .was respon- 
sible for conducting the actual training program of seminars and workshops, 
providing staff members, consultants and instructors, as required. • 

The project was conducted at three levels for school district person- 
nel. Superintendents and board members met in a one-day session to discuss 
current thinking on the roles of research and evaluation in school systems 
and their function to policy and decision-making. Research directors met 
in two three-day seminars for presentations and discussions of current 
problems and research techniques. Research and evaluation staff members 
attended two five-day training seminars to review statistical techniques, 
evaluation models, and data collection instruments. All presentations were 
planned to upgrade and update the capabilities of practicing skilled prof- 
essionals . 
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This project was an evaluation-research training program. The prime 
contractor was the Research Council of the Great City Schools , and North- 
western University, the subcontractor. 

The purpose of the project was to involve local program evaluators in 
training activities that would increase competencies toward evaluation and 
the consequent improvement of instructional programs at the local level. 
Northwestern University served as the subcontracting agency and .was respon- 
sible for conducting the actual training program of seminars and workshops, 

providing staff members, consultants and instructors, as required. • ' ' 

• • 

The project was conducted at three levels for school district person- 
nel. Superintendents and board members met in a one-day session to discuss 
current thinking on the roles of research and evaluation in school systems 
and their function to policy and decision-making. Research directors met 
in two three-day seminars for presentations and discussions of current 
problems and research techniques. Research and evaluation staff members 
attended two five-day training seminars to review statistical techniques, 
evaluation models, and data collection instruments. All presentations were 
planned to upgrade and update the capabilities of practicing skilled prof- 
essionals. 



INTRODUCTION 



The effectiveness and efficiency of a large- city school 
system depends upon a well -organized, well-trained staff of 
administrators working cooperatively. Information feedback 
from students, staff, community, as well as state and national 
agencies, require an increasingly alert organization* to collect 
and process information critical to the decision-making and policy- 
setting aspects of school administration. Opportunities for experience- 
sharing ana critical analyses of current problems, with, emphasis upon 
development of internal problem-solving expertise become important 
avenues to successful research-oriented operations. Decisions based 
upon field -gathered data must be made with some understanding of 
issues and methods involved. Boards of education and superintendents 
not only need to know what' decisions are being made. elsewhere in 
the country, but also the rationale for such, decisions and any diffi- 
culties for assessment of evidence presented. Research directors 
and research personnel need to be exposed to new ideas and techniques 
in such a manner that they can collectively evaluate and adapt these 
ideas to a level of practical value. Efforts to upgrade staff personnel 
qualifications in skill areas serve an important role in a local educa- 
tional agency' s attempt to improve the quality of curriculum and 
instruction, and practical applications of new ideas or methods are 
facilitated through the face-to-face communication of seminars 
because of the wide variety of experiences and systems backgrounds. 

Specific objectives of the Evaluation Training Project were as 
follows: 



. To improve school programs and evaluations in large cities 
by improving the technical competencies of individuals 
involved in making evaluations. 



. To develop a training program format for continual up- 
grading of staff personnel involved in gathering, analyzing, 

and interpreting data relating to administration and decision- 
making. 



. To provide opportunities for 
and promote inter-city coop 
efforts.' 



face-to-face information exchange 
e ration in research and evaluation 



Experts in various fields were brought into the project to 
inform and react to ideas or questions of school personnel. Topics 
covered by consultants and staff members encompassed numerous 
areas of interest (see meeting agendas. Appendix E). The consultants' 
presentations were appropriate and relevant to research and evaluation 
problems. Included in the seminar schedules were group tours to • 
such places as a research bureau, an MES school, and the SR A 
Development Center. Questionnaires were designed to allow seminar 
participants to express their opinions, point out special problems or 
difficulties, and offer recommendations for planning future sessions 
(see Tabulation of Results, Appendix F). 



Methods 
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The Prime Con tractor 

«— « - , -- r - .-it — f- 

The Research Council of the Great Cities Program for School 
Improvement is an organization of twenty (2.0) large city school districts; 
a not-for-profit corporation dedicated to the improvement of education 
in the Great Cities of the nation. Its purpose is to conduct studies of 
unique problems faced by the Great Cities in their effort to meet the 
comprehensive public school needs of their citizens, to coordinate 
projects designed to provide solutions to these problems, and to 
sponsor the carrying forward in practice of the results and findings 
of studies to promote school improvement in the cities. The affairs 
of the Research. Council are governed by the Board of Directors. 
Membership of the Board includes one member .of the Board of 
Education and the superintendent ■ from each participating city. 

The Board of Directors convenes twice each year. Between the 
semi-annual meetings of the Board of Directors, the Executive 
Committee is empowered to manage the affairs of the Research 
Council. In additional to seven (7) members elected by the Board 
of Directors, the Executive Committee includes the President, Vice- 
President, Secretary/Treasurer and the immediate past President of 
the Research Council. The Research Council staff, under the direction 
of the Executive Vice President, organizes and coordinates the study 
activities of the Council. Committees consisting of the staff represetit- 
atives from each city are organized to serve in the planning and continuation 
of various study areas that are of central concern to the Great Cities. 

The training program was conducted for the following member 
school districts of the Great Cities Research Council: Baltimore, 

Boston, Buffalo, Chicago, Cleveland, Detroit, Los Angeles, Memphis, 
Milwaukee, New York, Philadelphia, Pittsburgh, St. Louis, San Diego, 

San Francisco, and Washington, D. C. 



The Sub -contractor 



The location of Northwestern University, just minutes from the 
then existing Chicago office of the Research Council, was ideally located 
to provide maximum coordination in project efforts with the Research 
Council. Northwestern University's Dean of Education, B. J; Chandler, 
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and his assistants, Steven T. Kolbroolc and Emanual Hurwitz, were 
able to provide the channels through which project consultants and 
instructors were obtained. Also their practical experiences and 
proficiencies in the educational field minimized the time and effort 
required in locating appropriate resource personnel. 

• * 

* _ * • • • * 

The Evaluation Training Project was conducted at three levels 
for school systems’ personnel: 

1) One superintendent and board member from each system 

(see Appendix. A for list of participants) 

2) Directors of Research and Evaluation Divisions 

(see Appendix B for list of participants) 

• 3) Selected staff members of research and evaluation divisions 

(selected by Research Directors, see Appendix C 
for list of participants) 

A meeting for superintendents and board members was scheduled 
at Boston, May 2nd, through 4th, 1968, during the Spring Board of Director 
Meeting of the Research Council. 

Two three-day seminars were held for Research Directors, one 
in Evanston, Illinois (April 3rd through April 5th, 1968) and one in 
Riverdale, New York (October 14th through October 16th, 1968). 

Two five-day seminars were also held at Evanston for research 
staff personnel May 20th through May 24th, 1968, and November 18th 
through November 22nd, 1968. 

Project coordination was directed by Northwestern University's 
Dean of Education, - B. J. Chandler, with the aid of assistants Steven 
T. Holbrook and Emanual Hurwitz; and Alva R. Dittrick, Executive 
Vice President of the Research Council, and his staff. The Steering 
Committee, composed of the Research Directors from five of the Great 
Cities’ school districts, functioned as a planning committee, outlining 
content areas of current importance to research and evaluation (see 
Appendix D). Consultants and instructors were contracted by the 
Research Council after mutual agreement was reached with Northwestern 
.University.. , * 
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Specific objectives 
were as follows: 



of these training 



seminars for each group 



■ Superintendents, Board Mem bers 

. To focus the relationships of research and evaluation to ' 

administration decision-making. 

*' * • 

. To develop a sensitivity to data collection that results in 
asking research-oriented questions. 

Directors of Research 



. To develop more comprehensive knowledge of new concepts 
techniques. 

• * * , • * . 

Research Staff Personnel 



. To broaden research and evaluation experiences. 

. To provide an opportunity for staff personnel to upgrade 
their personal qualifications. 



The achievement of objectives of the Evaluation Training 
Project is difficult to measure directly or indirectly. It is hoped 
that the introduction of new ideas, methods, and techniques will 
upgrade the research and administrative capabilities of school • 
personnel who are frequently too preoccupied with duties to be as 
fully aware of recent developments as they should be. Furthermore, 
by design this project attempts to create a unique learning experience 
by bringing together field authorities and practicing school personnel. 
Such meetings not only assist school personnel with identifying and 
using appropriate outside consultative services, but also promotes 
the development of inter-city networks of communication that may 
■in some way benefit research- evaluation operations. For example, 
the author recalls a recent experience at a similar training session 
in Memphis, Tennessee. Representatives from two different cities 
were discussing the possibility of trading already developed optical- 
scanning forms for student attitude survey use, this after the 
representatives had become aware of each other’s project work 
disclosed during the sessions. In the example cited, each city 
would benefit through the saving cost, time, and trouble that would 
be entailed in separate development efforts along parallel paths. 

An important result of these training sessions lies in the 
identification of information necessary to propose a format for future 
training seminars or on-going seminars. For example, through 
polling the participants, future seminar criteria was identified: 

1) Specific topics of subject areas where research and 
evaluation personnel recognize a need to be better 
informed. 

V * ^ * 

2) Practicing school people willing to offer their services 
in planning or conducting workshops and serving a.s 
additional resource personnel. 

3) Areas of participant competencies that could be ’ v awn 
upon by consultants for assistance in preparing relevant 
presentations. 

• . • 

4) The most favorable time schedule for full participation. 

5) (Consultants with exceptional potential for use in future 

’ sessions. • 



Conclusions 



Decision -making by school superintendents and board members 
depends upon a proper utilization and perspective of research and 
evaluation tools. An understanding of methods; how they can be used 
to arrive at valid and justifiable decisions; and limitations imposed by 
research-evaluation designs or organization of data, are necessary 
implications that must be clarified before maximum potential in 
research-roriented operations can be achieved. The orientation 
session for superintendents and board members in the spring of 
1968 attempted to bring these issues to focus, so that research- 
evaluation programs are not an entity reserved only to specialists, 
but rather, a resource to be best utilized when administrators are 
keenly aware of relationships between research-evaluation activities 
and decision-making. 

* * ' ’ . * • .’*♦'*•* 

An equally important purpose of this project was the. inservice- 
training aspect involving Research Directors and research staff personnel. 
Many of these persons are aware of the need to increase their own research 
capabilities; a need that may be due to recent developments or findings 
in the experimental or developmental phase, or they may result from 
a lack of available resources as presently structured (i. e. , inappropriate 
university courses). In either case, inservice training sessions were 
structured to the needs of participants and attempts toward practical, 
relevant applications were a major consideration. 

Another primary concern in the Evaluation Training Project was 
the upgrading of back-up personnel in research and evaluation. Unless 
those individuals involved in gathering raw data are adequately prepared 
and familiar with new methods and techniques, they cannot possibly 
generate data in a desirable form or data with qualities upon which valid 
administration decisions may be made. An illustration of this project’s 
effort to inform and sharpen capabilities of research pei sonnel can be 
seen in the topic "Scaling Problems and Questionnaire Construction” 

(see A|^pendix E, Training Seminar for Research Staff Members, 

November 18th, 1968). Implied in this topic is a problem of many 
school districts development of valid, reliable surveys, question- 
naires, or test instruments for local use, with rea’sonable assurance 
that , sound decisions can be based upon findings. It is this kind of 
data -gathering skill development with which all research-evaluation 
and decision-making people must be concerned, for accuracy of 
interpretation (decision-making) depends upon the quality of original 
data gathered. 
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The concensus of staff participants’ opinions toward 
attendance of future seminars of this nature is quite clear; since 
these persons indicated what measures and directions should be 
pursued to improve the sessions, it is evident that their attitude 
is positive and supportive to continuation of inservice training 
activities. Best evidence to this effect was the large percentage 
of participants who indicated their willingness to help plan or 
conduct future sessions or act as resource personnel in their 
special fields. But the consensus of opinion for future training 
seminars revolves mainly around their ideas, for improving the . 
planning phase. Among the recommendations were: 

1) Greater participant involvement in initial planning. 

- 2 ) Consultant awareness of participant experiences/ 

competencies. 

3) Workshop technique preferred to straight lectures. 

4) Simultaneous sessions needed. 



5) Unstructured time desirable. 



Participants expressed an interest in selecting topics of 
interest, in proposing schedules of events, and in selecting possible 
resource personnel. They proposed that simultaneous sessions 
might be scheduled to allow a wider selection of topics -- topics 
to be dealt with in a workshop fashion to permit greater interaction 
of ideas precluded by the lecture-type presentation. However, 
participants indicated their belief that outside consultants should 
be tuned-in to the experience/competency level of the group (s) to 
maximize chances for theory and practice to emerge in practical, 
useful form. Unstructured time was cited as a desirable facet 
of seminars because of the opportunity to freely explore experiences, 
ideas, and materials of representatives from other cities’ systems. 

One of the specific objectives of the Evaluation Training 
Project was to develop a format for the continual upgrading of staff 
personnel. An important implication to the achievement of project 
objectives is a recognition of the transitory nature of planning variables 
involved: ... 
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Felt needs of participants, 

Subject areas ox interest (if different from M needs M )j 
Location of resources 

a) Consultants available 
# 

b) Accommodations available 

c) Materials available 
4) Time scheduling 

a) Participants 

• b) Consultants * . 

It becomes apparent that the greatest concern in meeting the 
objectives of this project is the problem of developing and coordinatin'* 
activities to meet expressed needs/intercsts of school personnel within 
the limitations of time and resources. 



1 ) 

2 ) 

3 ) 



R e c omm e nd a ti on 



The Evaluation Training Project achieved its objectives. 

School personnel from the three levels (superintendents and board 
members; research directors, research staff personnel) were 
assembled and introduced to new ideas and methods. The consultants 
and school personnel were provided the opportunity to react to each " 
other’s ideas, from theoretical to practical applications. School 
personnel from the Great Cities found the sessions useful, even to 
the extent that informal discussions on free time (dinner, etc. ) 
became a valuable medium of information exchange. 

Aside from other benefits of this project cited previously, 
continuation of this type of project is warranted because it can 
deliver the products efficiently within the duty-filled, schedules, of 
practicing school personnel, and the relationship established between 
theoreticians and educators promotes a guided evolution of knowledge - 
generating activities on the university level. The extension of public 
school influence into the institutions where teachers are being trained 
has important potential for future projects of this nature, because 
many of the problems of education and teacher preparedness can 
only be solved as public school and university personnel are brought 
together in common efforts of this type. 




10 



APPENDIX A . 



Board of Directors 

The Board of Directors of the Research Council includes one member 
of the Board of Education and the Superintendent of Schools from each 



[ member city. 

} CITY 
t Baltimore 
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BOARD MEMBER 
Mr. Francis D. Murnaghan, 
Mr. Paul F. McDevitt 
Mr. George F. Goodyear 
Mrs. Wendell E. Green 
Mr. George Dobrea 
Mr. Peter F. Grylls 
Mrs. Georgiana Hardy 
Mr. Edgar Bailey 
Mrs. Margaret Dinges 
Dr. Aaron Brown 
Mr. Richardson Dilworth 
Mrs. Maxine C. Aaron 
Mr. James E. Hurt, Jr. 
Mrs. Louise Dyer 
Mr. Adolfo- de Urioste 
Rev. Everett Hewlett 



- SUPERINTENDENT 

Jr. Dr. M. Thomas Goedeke 

(Proxy-Dr. V. S. Vavrina) 
• Dr. William Ohrenberger ■ 

Dr. Joseph Manch 

Dr. James F. Redmond 

Dr. Paul W. Briggs 

Dr. Norman C. Drachier 

Dr. Jack P. Crowther 

Dr. E. C. Stimbert 

Dr. Richard P. Gousha 

Dr. Bernard E. Donovan 

Dr. Mark R. Shedd 

Dr. S. P. Marlahd, Jr. 

Dr. William Kottmeyer 

Dr. Ralph Dailard 

Dr. Robert E. Jenkins 

Dr. William R, Manning 
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Great Cities Research Seminar 



New York City -- October 14-l6th, 1968 



Participants 



1. 


Claude Clapp 


Buffalo 


2. 


Gerald H. Moeller 


St. Louis 


3/ 


William Vogler 


San Diego 


4. 


Harold L. Weeks 


San Francisco 


5. 


Mildred Cooper 


Washington, D. C. 


6. 


Samuel D. McClelland 


Brooklyn, New York 


7. 


Jv W.ayne Wrights tone . • 


Brooklyn, New York. . . t • . 


8. 


Orlando F. Fur no * 


Baltimore 


9. 


Margaret M. Callahan 


Boston 


10. 


Malcolm Provus 


Pittsburgh 


li. 


John L. Hayman, Jr. 


Philadelphia 


12. 


O. Z. Stevens 


Memphis 


13. 


William Ashbaugh 


Milwaukee 


14. 


Emmett Moll . 


Milwaukee 


35. 


Joseph L. Mazur 


Cleveland 


16. 


Robert S. Lankton 


Detroit 


17. 


Everett W axman 


Los Angeles 


Consultants 




1. 


Desmond L. Cook 


Ohio State University 


2. 


Ralph Oravec 


Price Waterhouse & Co. , New York 


3. 


Donald Rappaport 


Price Waterhouse & Co. , Philadelphia 


4. 


Frank Leitner 


Price Waterhouse & Co. , New York 


5. 


Oliver Brown 


Price Waterhouse & Co. Philadelphia 




- 


Schools 


Coordinator s 




1. - 


Alva R. Dittrick 


Great Cities Research Council 


2. 


Jack I. Marcus sen 


Great Cities Research Council 


3. 


B. J. Chandler 


Nor thwe stern University 


4. . 


. Steven T. Holbrook 


Northwestern University 


5. 


Jon Peterson 


Northwestern University 



6 . 



Stuart Westerlund 



U. S. Office of Education 
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Baltimore: 

Boston: 

Buffalo: 

Chicago: 

* Cleveland:- - 

Detroit: 

Los Angeles: 

Memphis: 

Milwaukee: 

New York: 
Philadelphia: 

Pittsburgh: 

•St. Louis: 

San Diego: 

San Francisco: 
Washington, D. 



Research Training Seminar 



Mrs. Beverly'W. Eliinwood, Research Associate 
Mr.. N. Craig Cutter, Research Associate. 

Mr, Lawrence Hagerty, Research Assistant 
Mr. John Lo Conte, Research Assistant 

Mr. Ronald Banks, Director of Curriculum Evaluation 
Dv, Eugene Samter, Director of Finance and Research 

Mr. Douglas Stone, Director of Research 

Miss Marian Kilbane, Staff Assistant -.Research - 
Mrs. Halle Francies, Staff Assistant - Research 
Dr, Ofelia Hala.sa, Staff Assistant - Research 

Mr. George W. Jacobs, Assistant Director 
Dr. Ferdinand Galante, -Research Assistant 
Joseph Zubowski / Peter Monas 

Mr. Claude Stone, Supervisor /Evaluation and Research 

Mrs. Virginia Blanton, Research Assistant 
Gene Barlow 

Mr. G. Dwight Rowe, Coordinator of Educational Research 
Mr. Elfred Bloedel, Supervisor, Testing Service 
Dr. William Ashbaugh, Chairman - Staff Committee 
Dr. John Belton, Supervisor, Educational Testing 

Dr. Samuel McClelland, Assistant Director of Research 
Richard Turner 

Mr. Norman Wexler, Specialist in Design & Analysis 
•Mr. Ellery M. Pierson, Research Associate 
Edward E. Brown 

Mrs. Mary Jane Duda, Coordinator for Research for 
Instruction & Teacher Training 
Esther Kresh 

Mrs. Doris Mueller, Supervisor of Research 
David Mahan 

Mr. Thomas Crellin, T«. S, A. - ESEA, Evaluation 
Stuart Macnof sky / Joseph Ford 

Mr. Yvon O. Johnson, Research Assistant 

C. : Mrs. Josephina Ordonez, Research Associate 

Mr. William R. Manning 



APPENDIX D 



Orientation sessions for superintendents and board members 
will include ; 

(a) the uses of research for decision-making. 

(b) the organisation of a research and evaluation department 
and the most effective lines of communication. 

(c) means for supporting research and evaluation. 

(d) the role of research and evaluation in the entire system. 

The seminars for Research Directors are to include the following 
* topics: ' • • * • ' • 

(a) research and evaluation applications in support of 
administrative decision-making. 

(b) organization and management of a research and 
evaluation department; channels for communication 
with the remainder of the school system. 

(c) new developments, and problem areas in research 
and evaluation techniques (including method and design, 
criterion selection, measurement, and statistical 
treatment). 

Content of the staff training workshops will be divided into 
four major categories. These four major topic areas will 
be offered simultaneously in order that participating staff 
pe?:sonnel may concentrate in their areas of specialty. 

Within these four categories, the following material will 
be presented: 

(a) proposal development 

(b) design and analysis 

(c) ' stating objectives in terms of behavioral changes 

* • 

(d) instrument development 



(e) ’ tests and measurement 






(f) 



survey methods 



.(g) observation techniques 

(h) statistical procedures beyond descriptive levels ' 

(i) computer technology 



: 
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Tabuutxoks 0? quysncmta^fi - Groups A & 3, C, and 



i; 



€ro ttp__ jA_jt_B : . . . . 

1. L«Ci» or tu; xour eeiranar sessions ic tu bo ranked on a 3 point scale 
ranging fcrcm very high (:i> to vary low <5) 0 First rr.nl; it a&cor£*n» 

to ^ctgtiga. Then, ‘rank it according to the extent to vhich 
lt Jg2.yiii.g fa your expectation. 



A 

B 

C 

D 



Experimental end quasi- esper irx-ri fc s 1 design (Dr, C&mboi'n 
Xsport-wrifcJiig (Dr. Boras) * ‘ 

Assessment in education (Dr, Sechrest) 

Opinion polling (Dr; Bridges) 



Now, rank the four session. 3 : 



Very high, os 



oooooooo»oooooooo•o«ooc♦ooco , 



B. 



leaking «* 


1 


iStssr.tt, t.ar.sv. dt«» 


1 3 


r" r i 


**W4^M.US4M • 

!.... 5 


Expectation 


10 


8 • | 
1 


i -3 

t 


-o 


1 


Fulfil linen t • 


3 


4 


6 


6 


3 


Expectation | 


. 7 


5 


7 


3 ’ i 


r«j 


Fulfillment | 8 


5 


5 


3 


i 


Expectation 

• 


3 


1C 

. n - M . 


9 


1 


0 


Fulfillment 


. ° | 


I 3 


8 


10 


l 


Expectation 


LI 1 


1 5 j * 


8 


0 


Fulfillment 


5 ' I 


1 10 i 


t 4 


1 


l 



2. Do you fea.1 tbs participants should be involved in the planning of the** 
seminars? * 



yes.,. 2 



21 



no 



Xf so, in vast way; (numbers behind indicate similar replies) 

Indicating areas of interest - 14 
Scheduling of 'activitss -5 
Choosing resource persons « 3 

Directors plan more information for Title 1 programs - 2 
Consultants need to know experiences end cunpeteacies of participants ~ 2 
Kot i a.vaf..eo to tee concerns aad problems of the participants •» 2 
Each city should present at least one problem for analysis by the poop 
arid possible solutions to consider -2 
Asses omen t of capabilities in research needed 

i -d tali>.3 of consultants vjere too elementary 



Mood a preliminary ir.scfci.ag 
Participant experiences should be • f 5/veu 

,i 4 * ,,# 






2 



Should he sent prg~a&ssion qe cfitiostkaiire for program planning to asst n 
feedback of success end fail-ure to planners 
Let the participants serve, as the experts 

Please comment on the following aspects of the five day session: 

• * t * 

a. Facilities: 

adequate - good - fair - ok - satisfactory - 14 
very fine « very satisfactory -2 
. good «• fine ** 6 
very poor - 1 
iaeeH.ng room - poor 

need room in. the evening for discussions 
need a better blackboard 

more attention to chalkboard- and tape recorder. . • . • • 

phone ringing is disturbing 



b, Housing: 

adequate - fair - ok *• satisfactory ® 0 
very satisfactory « 1 
good - fine - 5 
poor - 5 
inexpensive 
. depressing hotel 

some people housed late 

Orringtoa staff curt and uncooperative 

comfortable 

need better mattresses 
c* Meals: 

ok ~ fair - average - 4 
good - fine - 5 

exceptionally fine «* very good - very satisfactory - excellent - 

poor - dull -** 3 

need variety end choice « 3 

luncheons should be less filling - too much - 5 

The Huddle - poor service 



d. Registration. 

confused a3 to hotel accomodations «• 6 

more directions and information at first needed ~ 3 

good - 3 

very satisfactory - very good - 2 . , . . 

fair 

need registration room. 

no table to receive, information upon arrival at hotel 
a printed list of information of places to go or see id desirable 
need earlier mixing end socialising of participants 
need to be advanced of beginning sessions 

seemed of little value since the hotel wasn't ready with rooms - ! 
not adequate 

pre»*scss5,cn letters not be cent tc superintendent but to iwxodiati 
encountered rudeness from registration desk. 






c » Organ! zation : 



f. 



very fine - well organised 
good - fine - 6 
ok - 



vary satisfactory - excellent - very good 



pre-planned 

consultants be briefed cn group needs 
lack of continuity in lectures 

need better advanced com&enic a l i on - suiter la! ahead of time - 4 
need to be more practical 

more suggestions and directions in written form 

division into smaller groups 

more care to organisational details 

3 days is sufficient • * 

too structured by non-participants ' 

Manner of Presentation (Lecture, workshop, etc,} 



workshop technique more rewarding - 7 
good -3 

very good - excellent -2 

oh, 

too much lecture - 4 
lecture too long 

best - lecture and workshop together - 2 
not always related to needs and abilities 
staff should be cn consultant level 
need change in manner of presentation 
poor ~ very dull - 2 

degree of success porportional to degree of participation by audience 



If you were planning another such program what changes would you effect ar 
and what would you retain? 

preplanning with participants on subjects - 5 
make presentations more relevant - 4 
more variety of activities in the day •* 3 
no long lectures •» 2 . 

retain professional consultants for problems defined by the group - 2 
3-4 day conference - 2 

include unstructured sessions for participants to exchange ideas - - 2 
increase pre-session communication with participants giving more 
information - 2 

more time devoted to sharing of experiences and problems 

hold conference at a different time of the year 

retain earns location . 

improve pre-planning - housing ai'rangemsnfcs and provisions of program 

schedule optional activities in the evening 






ho3.fl coriferes.ee ia an urban center 
coiratuaicate needs to consultants 
more specifics with, ragstfcl to Title X .programs 
have participants present area of specialisation 
delete “Srpsirinoatal and qua si-enperim&nt a 1 Design 1 * 

coordinator should keep tighter control and alter program as necessary to 
meet the needs of the participants 
Staff shoul.d be on consultant, level 



'• Group C 



T8FRSKETP - (The Sesd-Frass bating Scale to Measure end Evaluate 
Total Program) . 



‘ Hate the elements from this conferenca that you f eal you ecu use upon 
.your returii to your school system. 

Hark the three most valuable with a* ! M* M ’• 

Mark the tares least valuable with a "«** . 

Mark the three remaining with a M G n 

Indicate the major source of help for element : 



V - 2 

0-5 

M «» 5 


Improved coasaunicaticns between. Title X snd 
Non-Title I evaluators 


in o jwi *y 

• 9 • • 


# 

l 


* “ii 

0-2 
- - 2 


/restructuring of report writing 


3 


10 

4 

Mwr>um« >• 


•V - 2 

0 - 4 

« « 7 


Improved ability to plan research operation! 


5 


:r-y 
0-2 
- - 5 


Construction of Questionnaires 


3 

• ** 


12 


•!• •» 3 
0-3 
- - 7 


Use of consultants outside of education 


3 * 


5 


-b - 5 
0-6 
«— - 3 


*» 

Use ox uhobstrusive measures 


2 


7 


*b - 0 

0-7 

•*•*() 


* * ■ 1 

Requests for improved research facilities 
and services 


3 


0 

. 8 . 

0 


? - 2 
0-8 
« «• 4 


Use of regression « discontinuity method 


2 


* -12 
0-2 
- — I 


Improved inter-city contacts 


14 

'«*•*»*» ' *** ' 



0THS15: SaGSSSTXGSS 

Exchange of Ideas and Problem?. with other Participants 



CciKneat from a participant: This g u c . •» i: i onus i 1*0 is poorly designed cud 

ivapp srdluo to respond to* 



Grou p P 



1 * 



15 



2 . 



How should the next workshop be plerxied? 

1 i. By Northwestern University in collaboration, with Research 
Directors of the Great Cities 

2. By Northwestern University pursuant to a real, survey of all 
prospective participants • 

3. By a committee from the participants of the first workshop: 

2 a , selected by training project directors 
, Q b# considering of volunteers 

Assuming that finances are not a constraint, would you be will:; rig to 
participate in. cither the planning or conduct of the next r workshop? 

P lo no 21 2* yes 1 3« possibly 



If yes,- to what extent would you bs willing to participate? 

13 1. Both planning and conduct of the second workshop 



8 2. Planning only 



3. 



Ir project participants were used as resource persons for soz ss sessions, 
would you serve? 

6 1, no • 14 ^2. yes 



If yes, what aspects of your own experience are you willing to chare 
(success or problems) with workshop numbers? 

1, Organization end financing of research division 
2* Research in School Finance end State School Fiscal Policy 
3* Form designing for data processing 

4. Early childhood programs 

5. Dissemination of research information to teachers 

6. Recruitment of data-collecfcors or training of research assistants - problems 

7. Data reduction and processing - general procedures 

8. Coc&utors . 

9# Item analysis programs available 

10* Equating and scaling test forms 

IX# Design techniques -2 . j 

. 12# • Basic statistical methods 
13. Follow-up and Field Survey 

14o Data Card System for Title I participants - noncomputerized systems 
15. Techniques for involving school personnel in evaluating tasks" 

1$. Techniques for analyzing educational programs . . 

17. Successes in my program that I think might help others - 2 

18. Problems in my program for which I seek solutions - 2 

19. Solutions to practical problems .in administering research and evaluation 

procedures in a large city school system 

20. Sharing of problems or successes in Title I evaluations 



21, 


Administration of research 


22, 


lesign of 


: Separate School Studies 


23* 


Report Hr 


ifcing 


24. 


Graphical 


Presentation 


25* 


Planning 


ami implementing .research projects 


26. 


Evaluation (methods and techniques) 


27. 


Cosaroavity 


■ relations ia research 


28* 


Training 


research personnel 



4. Should future css'xn&rs * . 

$ Sti limited to a coinmos or single set of sessions for all 

participants? 



j3 2, Include simultaneous sesDions ca different topics? 



5« At what time during the year do you think subsequent conferences should he held? 
3 . during the summer months 



18 during the school year 



A. If during the eujncer s rank in order of preference (3 i3 high. I is low): 

June - 3 - (2), 2 - (X), 1 ~ (3> 

July - 3 » (1), 2 - (3), 1 - (1) 

August-3 - (2), 2 - (0), 1 - (3) 



B< 



If dtsriag the school year, in which months would it he most fee sills for 
you to attend? (list in order your first three choices) 

1st choice — Kov.-7, Ccto-5, Sept, -3, Bsc, -2, April-1 

2nd choice — Jsn,«5, I*ova.-3, Cct„-3, PeCc-2, Feb-, -2, April-1, March-1, Eay-1 
3rd choice — March -5, Feb* -5, Hay -3, Jsiw-X, April-1, June-1 



6, How long should the next conference last? 

9 one day 3 two days __8 three days 4 four days 

j , r " ,n,r '■ 

f ive days 

A* Keeping the answer in mind, in what kind of setting should the next 
conference ha held? 

_.JL~ Iu a secluded retreat 
.... 7 On a university campus 
, 12 In an urban center 



7 . Indicate your potential interest in the following topics by using a 4 point 
scale for each item, A ,, 4 M indicate high interest, a "1” will indicate a 
mild interest. Please specify anyone (in the space provided) that you 
think would ba appropriate as a resource person* 



Resource Person 



■ Topic 



Response^' 

D *<Vl 4 

4- (6), 3.*»<3}» 2- (4), X" (5) PrSSSS : -t5Ja‘" Regression 

4- (5)* 3- (7) s 2- (3) » X- (X) General ta&ex&xt Ul 

* # 

4- (X?) , 3«(4), 2- (2) , 1«(2) Design techniques 

a 

Test Construction: 



4- (8); 3- (3) , 2- (5) s 1~(2) Attitudes 

4** (7) 5 3~(7) , 2** (3) , l«(i) Tiaiovafciva Techniques 

4-<4), 3- (7)^. 2~(2)„ X-(4> . Ach« for disadvantaged 

p ate P roce ssin g: 

4- (7), 3- (5), 2- (2) 9 1“(4) Application to resource 



Dr. Cacpball (sore than 
. the technique) 

Horai Uaxler (cover Camp** 
bell & Stanley Designs) 



Lee Cronbsck 

Dr* Cassel (U.U.-hftteukse) 

Firin-ETS . Consultants . 

(Dr„ Tansice) 

S« Ksgia . . 

• • 

Dr* Baker - (U,W.**Madisori) 



4* (7), 3»* (2), 2- (4), l-<4) 

4-* (3), 3w(2), 2- (7) * l-(3) 
4~ (3) , 3- (3) 2~ (4) * 1“ (4) 
4-<7>, 3-<2>, 2"<4) 9 l-(3) 



PrograEsdttg Kora {foxier 

Administrative ; 

In terdepsr tsient al relations Ed Sur fmsit 



Public Relations 

Report Writing 
0 

Others : specify 

Interview and Polling Techniques 
Research Kcnsger&nt 
Observational Techniques of 
Teachers and Children 

PERT Chart Use 

Coordination of research 

(jSvsluation-Xnter-departEsntsl 



Ur; Bridges 
Desmond Cook 

Dr* Brimno Bueller- 
ier«pj.e 

Dr. Jac« bowita - 
U. of 111. 

Dr. Jacobowifcz 



*Kusaber in parenthesis indicates response 



Aft* , 1 1 *i fti rflSVi jl lit 1 1 ' ^ 



